Goto

Collaborating Authors

 Middle East


Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions

Neural Information Processing Systems

The Restricted Boltzmann Machine (RBM) is one of the simplest generative neural networks capable of learning input distributions. Despite its simplicity, the analysis of its performance in learning from the training data is only well understood in cases that essentially reduce to singular value decomposition of the data. Here, we consider the limit of a large dimension of the input space and a constant number of hidden units. In this limit, we simplify the standard RBM training objective into a form that is equivalent to the multi-index model with non-separable regularization. This opens a path to analyze training of the RBM using methods that are established for multi-index models, such as Approximate Message Passing (AMP) and its state evolution, and the analysis of Gradient Descent (GD) via the dynamical mean-field theory. We then give rigorous asymptotics of the training dynamics of RBMs on data generated by the spiked covariance model as a prototype of a structure suitable for unsupervised learning. We show in particular that RBMs reach the optimal computational weak recovery threshold, aligning with the Baik-Ben Arous-Pรฉchรฉ (BBP) transition, in the spiked covariance model.


Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks

Neural Information Processing Systems

We study the dynamics of stochastic gradient descent (SGD) for a class of sequence models termed Sequence Single-Index (SSI) models, where the target depends on a single direction in input space applied to a sequence of tokens. This setting generalizes classical single-index models to the sequential domain, encompassing simplified one-layer attention architectures. We derive a closed-form expression for the population loss in terms of a pair of sufficient statistics capturing semantic and positional alignment, and characterize the induced high-dimensional SGD dynamics for these coordinates. Our analysis reveals two distinct training phases: escape from uninformative initialization and alignment with the target subspace, and demonstrates how the sequence length and positional encoding influence convergence speed and learning trajectories. These results provide a rigorous and interpretable foundation for understanding how sequential structure in data can be beneficial for learning with attention-based models. Stochastic Gradient Descent (SGD) is the core optimization tool driving modern machine learning. Recent years have seen substantial progress in understanding its dynamics, particularly in two-layer networks [Saad and Solla, 1995, Mei et al., 2018, Chizat and Bach, 2018, Rotskoff and VandenEijnden, 2022, Sirignano and Spiliopoulos, 2020, Arnaboldi et al., 2023a]. While global convergence is qualitatively well-understood when the network is wide enough, quantitative results are scarcer. A particularly fruitful body of recent theoretical work addressing this gap has focused on deriving precise convergence rates for particular model classes on synthetic data, such as high-dimensional Gaussian single and multi-index models [Ben Arous et al., 2021, Abbe et al., 2022, 2023].


Optimal Spectral Transitions in High-Dimensional Multi-Index Models

Neural Information Processing Systems

We consider the problem of how many samples from a Gaussian multi-index model are required to weakly reconstruct the relevant index subspace. Despite its increasing popularity as a testbed for investigating the computational complexity of neural networks, results beyond the single-index setting remain elusive. In this work, we introduce spectral algorithms based on the linearization of a message passing scheme tailored to this problem. Our main contribution is to show that the proposed methods achieve the optimal reconstruction threshold. Leveraging a high-dimensional characterization of the algorithms, we show that above the critical threshold the leading eigenvector correlates with the relevant index subspace, a phenomenon reminiscent of the Baik-Ben Arous-Peche (BBP) transition in spiked models arising in random matrix theory.


World Cup picks for Brazil vs Morocco and Norway vs Japan with over bets and a draw prediction

FOX News

Pat McAfee wages war on Omaha's famous Jell-o shot bar after crew gets cold reception at College World Series NASCAR legend Tony Stewart calls mourning fans'a--holes' in tone-deaf rant about Kyle Busch Brewers' Jacob Misiorowski breaks brains and radar guns with hardest pitch ever by a starting pitcher US fans were out in full force ahead of the USMNT's first match of the 2026 FIFA World Cup MLB announces drive-in theater screenings of'The Sandlot' with live games and fireworks for July 4th California Democratic Party under fire for'you're not allowed to watch' World Cup post Victor Wembanyama isn't good or mature enough to be the face of the NBA -- at least not yet Trump praised for having'lots of energy' ahead of 80th birthday Trump calls Maine Democratic Senate candidate Graham Platner a'thug' Charter Space founder responds to critics' worries about SpaceX impact on market Rep. Byron Donalds shares his faith redemption story amid Florida gubernatorial run Iran's foreign minister says peace with US'has never been closer' GOP lawmaker says it's'really important' that US continues cartel crackdown Spencer Pratt's use of AI to boost campaign sparks debate FBI arrests first suspect on'most wanted fraudsters' list Brazil favored at -145 with the over at 2.5 +115, while Japan's tactical play could neutralize Haaland INSTANT REACTION FIFA World Cup Now reacts to USA's 4-1 dominant win over Paraguay Melissa Ortiz, Peter Crouch, Sacha Kljestan, Bob Bradley, Stu Holden, Brad Guzan and Mo Edu react to USA's 4-1 win over Paraguay. We are all jazzed up about the World Cup, right? I mean it is in our own backyard this year and the USA Men's National Team just won their first game with a dominant 4-1 victory over Paraguay. More importantly to me, we just won 1.35 units on the game because we took the over for it. I'm headed back to the pitch today for a couple of different plays.


Pat McAfee wages war on Omaha's famous Jell-o shot bar after crew gets cold reception at College World Series

FOX News

NASCAR legend Tony Stewart calls mourning fans'a--holes' in tone-deaf rant about Kyle Busch Brewers' Jacob Misiorowski breaks brains and radar guns with hardest pitch ever by a starting pitcher US fans were out in full force ahead of the USMNT's first match of the 2026 FIFA World Cup MLB announces drive-in theater screenings of'The Sandlot' with live games and fireworks for July 4th California Democratic Party under fire for'you're not allowed to watch' World Cup post Victor Wembanyama isn't good or mature enough to be the face of the NBA -- at least not yet Rep. Byron Donalds shares his faith redemption story amid Florida gubernatorial run Iran's foreign minister says peace with US'has never been closer' GOP lawmaker says it's'really important' that US continues cartel crackdown Spencer Pratt's use of AI to boost campaign sparks debate FBI arrests first suspect on'most wanted fraudsters' list Accused Charlie Kirk killer's attorneys seek to BLOCK death penalty Kayleigh McEnany: Capitalism isn't the big evil Bernie Sanders would have you believe OutKick Sports Pat McAfee wages war on Omaha's famous Jell-o shot bar after crew gets cold reception at College World Series McAfee says the general manager was unhappy he didn't call ahead and mocked his ability to pay for shots Dan Dakich asks how ESPN's relevance has changed since adding Pat McAfee. We've got drama at the College World Series, and it has nothing to do with baseball. Pat McAfee has waged war with Rocco's -- the famous Omaha-based bar known for its Jell-O shot challenge during the 12-day tournament. And by war, I mean McAfee stuffed the GM in a locker during a heated segment on his ESPN and YouTube show Friday afternoon. It was nowhere near what I thought it was going to be like, McAfee said of the crew's experience at the bar earlier this week.


Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions

Neural Information Processing Systems

The Restricted Boltzmann Machine (RBM) is one of the simplest generative neural networks capable of learning input distributions. Despite its simplicity, the analysis of its performance in learning from the training data is only well understood in cases that essentially reduce to singular value decomposition of the data. Here, we consider the limit of a large dimension of the input space and a constant number of hidden units. In this limit, we simplify the standard RBM training objective into a form that is equivalent to the multi-index model with non-separable regularization. This opens a path to analyze training of the RBM using methods that are established for multi-index models, such as Approximate Message Passing (AMP) and its state evolution, and the analysis of Gradient Descent (GD) via the dynamical mean-field theory. We then give rigorous asymptotics of the training dynamics of RBMs on data generated by the spiked covariance model as a prototype of a structure suitable for unsupervised learning. We show in particular that RBMs reach the optimal computational weak recovery threshold, aligning with the Baik-Ben Arous-Pรฉchรฉ (BBP) transition, in the spiked covariance model.


Fully autonomous drones have killed human soldiers for the first time

New Scientist

Fully autonomous drones with no human oversight have killed soldiers on the battlefield for the first time. This is according to a senior figure in the Ukrainian defence industry, marking a watershed moment in warfare. The one-off test involved 10 AI-controlled "Terminator" drones on the front line of the Ukraine war. "We tried it," says drone-maker Alexander Kokhanovskyy, who supplied the technology and spoke to at a press event hosted by the Ukrainian embassy. We never implemented it [more widely]." The test took place two years ago and involved quadcopter drones that were programmed to fly towards the front line, cover between 3 and 5 kilometres over around 10 minutes and then engage "Terminator mode", in which an AI model searches for and intercepts targets. "We just launch it and we know everything will be dead - everything that will be found there in this particular area will be dead," says Kokhanovskyy. "There is no connection to the drone at all, you cannot see the video, ...


World's shark attack hotspots revealed: As a great white is spotted in the Mediterranean, experts reveal the areas where you're most likely to be bitten

Daily Mail - Science & tech

'Record the faces': Tense moment NBA boss gives VERY honest take on Trump attending Knicks game Leaked transcript of UNAIRED 60 Minutes interview exposes REAL reason'callous' CBS star Scott Pelley'deserved to be fired' Disgraceful texts'hot' teacher sent boy, 17, who she had illegal sex with where she moaned about her HUSBAND Everyone always said I cleared my throat a lot. But then I developed shoulder pain and doctors discovered the sinister cause... the world's deadliest cancer. Don't leave it too late like I did Outrage as Netanyahu is caught SPYING on Trump's Iran negotiators... as JD Vance reveals a chilling truth about Israel White couple gave birth to'non-Caucasian' baby. Parents were told son, 7, had ADHD... not realizing he was battling terrifying disease that has now left him BLIND'Great' mom, 32, tried to gas herself and her three young kids to death after inviting them to'popcorn sleepover' in car, prosecutors allege Medical student, 24, died by suicide in his white coat a day after he was suspended for alleged'inappropriate' behavior towards female patient, lawsuit alleges, as his heartbreaking goodbye note to parents is revealed Karmelo Anthony's parents seen leaving the courtroom in tears just before son's defense team pulls shock move Grim-faced former Louisiana mayor Misty Roberts arrives in court for sentencing after being found guilty of having sex with son's teenage friend Mother died during tummy tuck and Brazilian butt lift after clinic staff failed to hold'slow' elevator for EMTs, report alleges Gaming influencer Alex Cimo dies'very suddenly' aged 32 just a month after'refusing to accept his fate' The porn-fuelled fantasy middle-class husbands are desperate to try with their wives... and it almost always ends in divorce: JANA HOCKING All the backstage gossip from Miami Swim Week: Insider exposes'catty' VIP's diva demands... STEALING... and'morbidly embarrassing' celeb moment everyone is whispering about Girl, 13, mistakenly told she was DYING after Oregon hospital staff made jaw-dropping surgical mistake, parents' $17m lawsuit alleges Mother's final words before she was shot dead'by new husband' in front of her two young children'They have a problem with my country': Africa's best referee, who was denied entry to the US and will miss the World Cup, speaks out and insists he had a valid visa Furious dad films his partner in bed with his 19-year-old son: You've seen the viral video - now all three tell the Daily Mail what REALLY happened in the scandal gripping Australia World's shark attack hotspots revealed: As a great white is spotted in the Mediterranean, experts reveal the areas where you're most likely to be bitten The world's shark attack hotspots have been revealed, after a g reat white shark was spotted in the Mediterranean Sea. The enormous predator was recorded between Sicily and Tunisia, in what is believed to be the first ever footage captured of an adult great white in the area.


Great white shark is recorded underwater in the Mediterranean for the first time ever

Daily Mail - Science & tech

'Record the faces': Tense moment NBA boss gives VERY honest take on Trump attending Knicks game Leaked transcript of UNAIRED 60 Minutes interview exposes REAL reason'callous' CBS star Scott Pelley'deserved to be fired' Disgraceful texts'hot' teacher sent boy, 17, who she had illegal sex with where she moaned about her HUSBAND Everyone always said I cleared my throat a lot. But then I developed shoulder pain and doctors discovered the sinister cause... the world's deadliest cancer. Don't leave it too late like I did Outrage as Netanyahu is caught SPYING on Trump's Iran negotiators... as JD Vance reveals a chilling truth about Israel White couple gave birth to'non-Caucasian' baby. Parents were told son, 7, had ADHD... not realizing he was battling terrifying disease that has now left him BLIND Medical student, 24, died by suicide in his white coat a day after he was suspended for alleged'inappropriate' behavior towards female patient, lawsuit alleges, as his heartbreaking goodbye note to parents is revealed Karmelo Anthony's parents seen leaving the courtroom in tears just before son's defense team pulls shock move Grim-faced former Louisiana mayor Misty Roberts arrives in court for sentencing after being found guilty of having sex with son's teenage friend Mother died during tummy tuck and Brazilian butt lift after clinic staff failed to hold'slow' elevator for EMTs, report alleges Gaming influencer Alex Cimo dies'very suddenly' aged 32 just a month after'refusing to accept his fate' 'Great' mom, 32, tried to gas herself and her three young kids to death after inviting them to'popcorn sleepover' in car, prosecutors allege The porn-fuelled fantasy middle-class husbands are desperate to try with their wives... and it almost always ends in divorce: JANA HOCKING Meghan Markle's As Ever website has had'less than 400,000 US visitors' since January - as Duchess launches collaboration with a lifestyle influencer to plug her products Nashville's most-hated influencer sparked outrage with sick posts about teen girl who vanished into the woods after a party... now his incredible life of luxury is unraveling Girl, 13, mistakenly told she was DYING after Oregon hospital staff made jaw-dropping surgical mistake, parents' $17m lawsuit alleges Mother's final words before she was shot dead'by new husband' in front of her two young children'They have a problem with my country': Africa's best referee, who was denied entry to the US and will miss the World Cup, speaks out and insists he had a valid visa Massive twist in JPMorgan'sex slave' case as accuser unveils NEW dossier of wild claims: 'The story is about to change dramatically' A great white shark has been spotted underwater in the Mediterranean for the first time ever. Divers from Healthy Seas were removing ghost nets on an offshore shipwreck between Sicily and Tunisia when they spotted the predator.


Few-shot Cross-country Generalization of Tabular Machine Learning and Foundation Models for Childhood Anemia Prediction under Distribution Shift

arXiv.org Machine Learning

Background Childhood Anemia affects an estimated 40% of children aged 6-59 months globally and arises from heterogeneous nutritional, infectious, and socioeconomic factors that vary substantially across settings. This variability challenges the generalizability of predictive machine learning models, which often degrade under cross-population or temporal shifts. We investigated the utility a modern transformer-based tabular foundation model (TabPFN) as a complementatry framework with respect to supervised classical machine learning methods across diverse country contexts, with particular attention to data-scarce settings where surveillance capacity is most limited. Methods We conducted a multi-country prediction study using Demographic and Health Surveys (DHS) children's recode data from 16 countries spanning Africa, Asia, Latin America, the Caucasus, and the Middle East. The harmonized analytic cohort comprised of (n = 68,856)children aged 6-59 months with valid hemoglobin measurements. Anemia was defined using WHO age and altitude-adjusted thresholds and treated as a binary outcome. We trained Logistic Regression, XGBoost, and LightGBM models using standard supervised learning, and evaluated TabPFN v2.6 in an in-context learning setting. Performance was assessed using Area Under the Receiver Operating Characteristic Curve (AUC-ROC) and other standard classification metrics, with calibration evaluated via Brier score and expected calibration error (ECE). Uncertainty in performance estimates was quantified using bootstrap resampling to derive 95% confidence intervals. Robustness was assessed in a few-shot learning setting. Cross-population generalization was examined using leave-one-country-out (LOCO) validation and reverse-LOCO experiments to assess directional transferability. Subgroup analyses were conducted across five demographic strata: child age group, sex, maternal education, residence type, and household wealth quintile. Feature importance was assessed using standard linear and tree-based explainer SHAP values for the three supervised models and an adapted version of SHAP for TabPFN, aggregated across countries and examined at the country level. TabPFN also yielded the best probabilistic calibration across all 16 countries, achieving the lowest mean Brier score (0.203) and Expected Calibration Error (ECE = 0.042) of all models evaluated; LightGBM and Logistic Regression exhibited the greatest miscalibration, particularly at higher predicted probabilities. Under full-data conditions, within-country discrimination was moderate across all models (AUC-ROC 0.59-0.76) Under LOCO validation, performance declined modestly (AUC-ROC 0.58-0.69) Reverse-LOCO analyses revealed asymmetric and directional transferability, with epidemiologically diverse populations serving as more informative training sources and certain target populations remaining persistently difficult to predict regardless of model or training data.